IMAGE GENERATING SYSTEM AND PROGRAM 

Technical Field 

The present invention relates to an image generating 
5 system and program. 

Background of Art 

In the prior art , there is known an image generating system 
which can generate an image viewable from a given viewpoint in 

10 an object space that is a virtual three-dimensional space. Such 
an image generating system is very popular as a system which 
can realize a so-called virtual reality. If the image generating 
system is for a racing game, a player can enjoy a three- 
dimensional game by controlling and running a racing car (or 

15 object) in the object space to compete with the other racing 
cars controlled by the other players or computer. 

However, an image generated by the prior art image 
generating system will not be focused depending on the distance 
from a viewpoint as in the actual human view. Thus, the 

20 representation is such that all the subjects within the image 
are in focus. 

The image in which all the subjects ranging between the 
short distance and the long distance are in focus is abnormal 
in our daily life. 
25 To pursue the reality, it is desirable that an image is 

generated in which a focus is regulated depending on the 
distance between the viewpoint and an object or the direction 



1 



of sight line. However, the processing load will be increased 
if a def ocused image is generated by calculating the distance 
between an individual object and the viewpoint within a game 
space to process the degree of defocusing for each object. 
5 In an image generating system which must generate an image 

corresponding to a real-time variable viewpoint using its 
limited hardware resource, it is important that a focused image 
as in the actual world can be generated using less processing 
load. 

10 

Disclosure of Invention 

To overcome such a problem in the prior art, an object 
of the present invention is to provide an image generating 
system and program which can generate a focused image as in the 

15- actual world with less processing load. 

To this end, the present invention provides an image 
generating system for generating an image, comprising: 
alpha-value setting means which sets an alpha-value for each 
pixel to a value corresponding to a depth value of the each pixel 

20 in an original image; and means for synthesizing the original 
image with a def ocused image corresponding to the original image 
based on the alpha- value set for each pixel. The present 
invention al£o provides a computer-usable information storage 
medium including a program for realizing the above - described 

25 means . The present invention further provides a computer -usable 
program (including a program embodied in a carrier wave) which 
includes a processing routine for realizing the above - described 
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means . 

According to the present invention, the original image 
is synthesized with the defocused image based on the set 
alpha-value corresponding to the depth value for each pixel in 

5 the original image. Thus, the synthesis ratio and the like in 
the defocused image is, variable depending on the depth value. 
Therefore, the depth of field may be represented, for example. 

The alpha ( a ) -value is information stored in association 
with each pixel and may be information other than color 

10 information, for example. The depth value may be increased as 
the pixel moves faraway from or close to the viewpoint (or 
virtual camera). The technique of synthesizing the original 
image with the defocused image may be any of various known 
techniques. The synthesizing process using the alpha-value is 

15 not limited to the alpha - blending . 

In the image generating system, information storage 
medium and program according to the present invention, the 
alpha-value setting means may set the alpha-value for the each 
pixel so that a synthesis ratio of the defocused image becomes 

20 higher for a pixel having a larger difference between a focal 
depth value of a virtual camera and the depth value of the each 
pixel. 

In such a manner, such an image that is defocused as it 
is spaced faraway from the focus can be generated. 
25 if the synthesis ratio of the defocused image increases 

as the alpha-value increases, the alpha-value may be increased 
as the difference between the depth value of the pixel and that 
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of the focus increases. On the contrary, if the synthesis ratio 
of the defocused image increases as the alpha-value decreases, 
the alpha-value may be decreased as the difference between the 
depth value of the pixel and that of the focus increases. 
5 In the image generating system, information storage 

medium and program according to the present invention, a range 
of depth of field or magnitude of defocusing effect may be 
variably controlled by varying a corresponding relationship 
between the depth value of the each pixel and the alpha -value. 
10 Thus, an image in which the range of depth of field or 

the magnitude defocusing effect is variable can be generated 
merely by varying the corresponding relationship between the 
depth value and the alpha- value. This improves the variety in 
the representation of image. 
15 The range of depth of field is not limited to any particular 

range, but may be a range between the nearest and farthest points 
in which a focused clear image may be obtained. For example, 
the range of depth of field may be defined to be a range in which 
the alpha -value becomes smaller than a given value or larger 
20 than a given value. The control of the magnitude of the 
defocusing effect may be realized, for example, by utilizing 
the same depth value varying the magnitude of the alpha- value. 

The present invention further provides an image 
generating system for generating an image, comprising: 
25 alpha-value setting means which sets an alpha-value for a pixel 
having a depth value located in an area AR1 between Zl and Z2 
as ai, the alpha- value for a pixel having a depth value located 
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in an area AR2 between Z2 and Z3 as a2, ... . , and subsequently 
the alpha-value for a pixel having a depth value located in an 
area ARi between Zi and Zi + 1 as ai; and means for drawing an 
image based on the alpha- value set by the alpha- value setting 
5 means. The present invention further provides a computer -usable 
information storage medium including a program for realizing 
the above- described means . The present inventioii further 
provides a computer-usable program (including a program 
embodied in a carrier wave) which includes a processing routine 

10 for realizing the above - described means. 

According to the present invention, the alpha-values for 
the pixels located at the areas ARI, AR2 , to ARi are set into 
ai, <X2, to ai, respectively. The alpha-values CC1, <X2, to 
ai thus set are used to draw the image. Thus, the alpha- value 

15 may simply be set depending on the associated depth value. The 
corresponding relationship between the depth value and the 
alpha- value may also variably be controlled in a simple manner . 

The drawing process based on the alpha -value set by the 
present invention is not limited to the synthesizing of the 

20 original image with the defocused image. 

In the image generating system, information storage 
medium and program according to the present invention, the 
alpha- value setting means may update: the alpha-value of an 
pixel located in a first direction from an object OBI having 

25 the depth value set as Zl by drawing the object OBI in a drawing 
area of an original image; the alpha-value of an pixel located 
in the first direction from an object OB2 having the depth value 
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set as Z2 by drawing the object OB2 in the drawing area of the 
original image; .... . and subsequently the alpha - value of an pixel 
located in the first direction from an object OBi having the 
depth value set as Zi by drawing the object OBi in the drawing 
5 area of the original image. 

In such a manner, the alpha-value can be set depending 
on the depth value through a light and high-speed process of 
only drawing objects. 

The first- direction may be one that extends farther or 
10 nearer as viewed from the object. 

In the image generating system, information storage 
medium and program according to the present invention, the 
alpha-value setting means may update only a bit to be updated 
by masking a bit not to be updated when an object is drawn in 
15 the drawing area of the original image to update the alpha - 
value . 

In such a manner, the alpha-value can be updated in bit 
unit. It is further desirable that the color or other 
information for each pixel is masked on updating the alpha - 
20 value. 

In the image generating system, information storage 
medium and program according to the present invention, the 
alpha- value setting means may update: an n-th bit in the 
alpha-value of a pixel located in a first direction from an 
25 object OBI' having the depth value set as Zl ' into a first level 
by drawing the object OBI ' in a drawing area of an original image, 
and the n-th bit in the alpha- value of a pixel located in the 
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first direction from an object OBI having the depth value set 
as Zl and located in the first direction from the object OBI' 
into a second level by drawing the object OBI in the drawing 
area of the original image; an (n+1) - th bit in the alpha- value 
5 of a pixel located in the first direction from an object OB2 ' 
having the depth value set as Z2 ' into the first level by drawing 
the object 0B2 ' in the drawing area of the original image, and 
the (n+l)-th bit in the alpha-value of a pixel located in the 
first direction from an object OB2 having the depth value set 

10 as Z2 and located in the first direction from the object OB2 ' 
into the second level by drawing the object OB2 in the drawing 
area of the original image; and an (n+i-l) -th bit in the 

alpha-value of a pixel located in the first direction from an 
object OBi' having the depth value set as Zi ' into the first 

15 level by drawing the object OBi' in the drawing area of the 
original image, and the (n+i-l)-th bit in the alpha-value of 
a pixel located in the first direction from an object OBi having 
the depth value set as Zi and located in the first direction 
from the object OBi' into the second level by drawing the object 

20 OBi in the drawing area of the original image. 

The first direction may be one that extends farther or 
nearer as viewed from the object. 

The present invention further provides an image 
generating system for generating an image, comprising: means 

25 for mapping a texture on an object by a texel interpolation 
method; and means for generating a defocused image for an 
original image by setting the original image as a texture and 
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shifting a texture coordinate of the object when the texture 
is mapped on the object by the texel interpolation method. The 
present invention further provides a computer-usable 
information storage medium including a program for realizing 
5 the above - described means. The present invention further 
provides a computer - usable program (including a program 
embodied in a carrier wave) which includes a processing routine 
for realizing the above - described means. 

- According to -the present invention, the defocused image 

10 can be generated through a simplified process by effectively 
using the texel interpolation method. 

The texel interpolation method is not limited to any 
particular form, but may be a method in which the image 
information of a pixel can be obtained by interpolating the 

15 image information of a texel, for example, such as the bi- linear 
filtering or tri-linear filtering method. 

In the image generating system, information storage 
medium and program according to the present invention, the 
defocused image of the original image may be generated by 

20 shifting the texture coordinate by a value smaller than one 
texel. 

In the image generating system, information storage 
medium and program according to the present invention, the 
texture coordinates may be shifted to a first shift direction 
25 to perform the texture mapping by the texel interpolation method 
and then the texture coordinates may be shifted to a second shift 
direction to perform the texture mapping by the texel 



interpolation method. 

Thus, a more ideal defocused image can be generated. 

It is desirable that the first and second shift directions 
are opposite to each other. It is further desirable that a 
5 plurality of shift sets of the first and second shift directions 
are repeated. 

In the image generating system, information storage 
medium and program according to the present invention, the 
object may be a virtual object having a shape substantially 
10 equal to a shape of a defocused area. 

If the entire image is to be defocused, it is desirable 
that the objects and defocused area have the same shape as that 
of the screen. However, only part of the screen area may be 
defocused . 

15 

Brief Description of Drawings 

Fig. 1 is a block diagram of an image generating system 
according to this embodiment. 

Fig. 2 illustrates a technique of setting an alpha -value 
20 corresponding to Z-value (depth value) and using the set 
alpha-value to synthesize a defocused image with an original 
image. 

Figs. 3A and 3B show examples of the original and defocused 
images . 

25 Figs. 4A and 4B illustrate a technique of varying the 

corresponding relationship between Z-values and alpha-values 
to control the range of depth of field. 
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Figs. 5A and 5B illustrate a technique of varying the 
corresponding relationship between Z-values and alpha-values 
to control the defocusing effect . 

Figs. 6A, 6B and 6C illustrate a technique of 
5 sectionalizing areas based on the Z-values to set an alpha- 
value for each of the sectionalized areas . 

Figs. 7A, 7B and 7C illustrate a technique of updating 
the alpha- value for a farther pixel in an object by drawing the 
ob j ec t . 

10 Figs. 8A, 8B and 8C illustrate the details of another 

alpha -value updating technique. 

Figs. 9A, 9B, 9C and 9D illustrate the details of still 
another alpha -value updating technique. 

Fig. 10 illustrates the texture mapping in the bi-linear 
15 filtering method. 

Fig. 11 illustrates a technique of generating a defocused 
image by effectively using the bi-linear filtering method. 

Fig. 12 illustrates another technique of generating a 
defocused image by effectively using the bi-linear filtering 
20 method. 

Figs. 13A and 13B illustrate a principle of generating 
a more defocused image through the interpolation in the bi - 
linear filtering method. 

Figs. 14A and 14B illustrate another principle of 
25 generating a more defocused image through the interpolation in 
the bi-linear filtering method. 

Fig. 15 i is a flowchart illustrating the details of a 
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process in this embodiment. 

Fig. 16 is a flowchart illustrating the details of an 
alpha-plane generating process . 

Fig. 17 is a hardware arrangement which can realize this 
5 embodiment . 

Figs. 18A, 18B and 18C show various system configurations 
to which this embodiment can be applied. 

Best Mode for Carrying Out the Invention 
10 A preferred embodiment of the present invention will now 

be described with reference to the drawings. 

1 . Layout 

Fig. 1 shows a block diagram of this embodiment. In this 
15 figure, this embodiment may comprise at least a processing 
section 100 (or a processing section 100 with a storage section 
170 or a processing section 100 with a storage section 170 and 
an information storage medium 180) . Each of the other blocks 
(e.g., control section 160, display section 190, sound output 
20 section 192, portable information storage device 194 and 
communication section 196) may take any suitable form. 

The processing section 100 is designed to perform various 
processings for control of the entire system, commands to the 
respective blocks in the system, game processing, image 
25 processing, sound processing and so on. The function thereof 
may be realized through any suitable hardware means such as 
various processors (CPU, DSP and so on) or ASIC (gate array or 
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the like) or a given program (or game program) . 

The control section 160 is used to input operational data 
from the player and the function thereof may be realized through 
any suitable hardware means such as a lever, a button, a housing 
5 or the like . 

The storage section 170 provides a working area for the 
processing section 100, communication seiction 196 and others . 
The function thereof may be realized by any suitable hardware 
means such as RAM or the like. 

10 The information storage medium (which may be a 

computer -usable storage medium) 180 is designed to store 
information including programs, data and others. The function 
thereof may be realized through any suitable hardware means such 
as optical memory disk (CD or DVD) , magneto -optical disk (MO) , 

15 magnetic disk, hard disk, magnetic tape, memory (ROM) or the 
like. The processing section 100 performs various processings 
in the present invention (or this embodiment) based on the 
information that has been stored in this information storage 
medium 180. In other words, the information storage medium 180 

20 stores various pieces of information (or programs and data) for 
realizing (or executing) the means of the present invention (or 
this embodiment) which are particularly represented by the 
blocks included in the processing section 100. 

Part or the whole of the information stored in the 

25 information storage medium 150 will be transferred to the 
storage section 170 when the system is initially powered on. 
The information stored in the information storage medium 180 
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may contain at least one of program code set for processing the 
present invention, image data, sound data, shape data of obj ects 
to be displayed, table data, list data, information for 
instructing the processings in the present invention, 
5 information for performing the processings according to these 
instructions and so on. 

The display section 190. is to output an image generated 
according to this embodiment and the function thereof can be 
realized by any suitable hardware means such as CRT, I*CD or HMD 
10 (Head-Mount Display) . 

The sound output section 192 is to output a sound generated 
according to this embodiment and the function thereof can be 
realized by any suitable hardware means such as speaker. 

The portable information storage device 194 is to store 
15 the player's personal data and save data and may be take any 
suitable form such as memory card, portable game machine and 
so on. 

The communication section 196 is designed to perform 
various controls for communication between the game system and 

20 any external device (e.g. , host device or other image generating 
system) . The function thereof may be realized through any 
suitable hardware means such as various types of processors or 
communication ASIS or according to any suitable program. 

The program or data for executing (or executing) the means 

25 in the present invention (or this embodiment) may be delivered 
from an information storage medium included in a host device 
(or server) to the information storage medium 180 through a 
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network and the communication section 196. The use of such an 
information storage medium in the hose device (or server) falls 
within the scope of the invention. 

The processing section 100 further comprises a game 
5 processing section 110, an image generating section . 130 and a 
sound generating section 150. 

The game processing section 110 is designed to perform 
various processes such as coin (or charge) reception, setting 
of various modes, game proceeding, setting of scene selection, 
10 determination of the position and rotation angle (about X-, Y- 
or Z-axis) of an object (one or more primitive faces), movement 
of the object (motion processing) , determination of the 
viewpoint (or virtual camera's viewpoint) and view angle (or 
rotational angle of the virtual camera) , arrangement of an 
15 object (or mapped object) within the obj ect space , hit checking, 
computation of the game results (or scores), processing for 
causing a plurality of players to play in a common game space, 
various game computations including game -over and other 
processes, based on operational data from the control section 
20 160 and according to the personal data, saved data and game 
program from the portable information storage device 194. 

The image generating section 130 is to perform various 
image processings according to the commands or the like from 
the game processing section 110. For example, the image 
25 generating section 130 may generate an image viewable from the 
virtual camera (or viewpoint ) within the obj ect space and output 
it toward the display section 190. The sound generating section 
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150 is to perform various sound processings according to the 
commands and the like from the game processing section 110. The 
sound generating section 150 may generate various sounds such 
as BGMs , sound effects, voices or the like and output them toward 
5 the sound output section 192. 

All the functions of the game processing section 110 and 
image, sound processing sections 130, 150 may be realized by 
any suitable hardware means or according to the program. 
Alternatively, these functions may be realized by both the 
10 hardwares means and program. 

The image generating section 130 comprises a geometry 
processing (or three-dimensional calculation) section 132, an 
alpha- value setting section 134, a def ocused - image generating 
section 136 and a drawing (or rendering) section 140. 
15 The geometry processing section 132 is designed to perform 

various geometry processings (three-dimensional calculations) 
such as coordinate transformation, clipping, perspective 
transformation and light source calculation. Object data (such 
as shape data including object vertex coordinates, vertex 
20 texture coordinates, brightness data and the like) obtained 
after the geometry processing (or perspective transformation) 
are then saved in a main memory 172 of the storage section 170. 

The alpha-value setting section 134 is designed to set 
the alpha-value for each pixel (which is information stored in 
25 association with each pixel and may be information other than 
the color information) into a value corresponding to the Z- 
value (which is a depth value in a broad sense) for each pixel 
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in an original image (e.g., perspec tively transformed image) . 
The alpha- value setting section 134 can variably control the 
range of depth of field and defocusing effect by varying the 
corresponding relationship between the depth value for each 

5 pixel and the alpha-value. Moreover, the alpha-value setting 
section 134 is to set the alpha-value for each pixel such that 
as that pixel is farther from the focus of the virtual camera, 
the alpha-value thereof will increase (or, in a broad sense, 
the synthesis ratio of the defocused image will increase) . 

10 The defocused- image generating section 136 is designed 

to generating a defocused (or most defocused) image obtained 
from an alpha- synthesis (e.g., alpha-blending, alpha-addition, 
alpha- subtraction, translucency processing or the like) with 
the original image. 

15 More particularly, this embodiment effectively uses the 

bi - linear filtering method (which is , in a broad sense, a texel 
interpolation method including the tri- linear filtering method 
or the like) for texture mapping to generate a defocused image. 
• In other words, the defocused- image generating section 136 is 

20 to set the original image as a texture and to shift (or slide 
or move) the texture coordinates of an object (which is a virtual 
object having the same shape as that of a defocused area) , for 
example, by a value smaller than one texel when the texture is 
mapped on the object through the bi- linear filtering method. 

25 Thus, the defocused image to be synthesized with the original 
image can be generated through a simple process in which the 
texture coordinates are only shifted. 
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The drawing section 140 is designed to draw an object 
viewable from the virtual camera within the object space, based 
on the object data, texture and so on. 

The drawing section 140 comprises a texture mapping 
5 section 142, an alpha- synthesis section 144 and a hidden- 
surface removal section 146. 

The texture mapping section 142 is designed to map a 
texture stored in a texture storage section 176 on an object 
(consisted of primitive faces such as polygons, free -form 
10 surfaces or the like) through the bi - linear filtering method 
(texel interpolation method) or a point sampling method. 

The alpha - synthesis section 142 is designed to synthesize 
the original image with a defocused image generated by the 
defocused- image generating section 136, based on the alpha - 
15 value (or A- value) set for each pixel by the alpha- value setting 
section 134. For example, if the alpha - synthesis is the 
alpha -blending, the synthesis of the original image with the 
defocused image will be performed as represented by the 
following formulas: 
20 R Q ' - ( 1 - Ct ) x r x + a x r 2 ( 1 ) 

G Q = (l - a) x Gl + a x g 2 (2) 
Bp (l - a) x Bl + a x b 2 (3) 
where R 1# G 1# B x are R, G and B color (or brightness) components 
in the original image which has previously been drawn in a frame 
25 buffer 174; R 2 , G 2 , B 2 are R, G and B color components in the 
defocused image generated by the defocused- image generating 
section 136; and R 0 , G Q , B Q are output images generated through 
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the alpha -blending . 

The hidden- surf ace removal section 144 uses a Z buffer 
178 (Z-plane) stored the Z-value (depth value) to erase the 
hidden surface according to the Z-buffer algorithm. This 
5 embodiment uses the function of the hidden - surf ace removal 
section 144 to update the alpha-value on drawing the virtual 
object. 

The image generating system of this embodiment may be 
dedicated for a single-player mode in which only a single player 
10 can play the game or may have a multi-player mode in which a 
plurality of players can play the game. 

If a plurality of players play the game, only a single 
terminal may be used to generate game images and sounds to be 
provided to all the players. Alternatively, a plurality of 
15 terminals interconnected through a network (transmission lien 
or communication line) may be used in the present invention. 

2. Features of this embodiment 
2.1 alpha ( a )- synthesis 

20 As shown by El in Fig. 2, this embodiment sets alpha - 

values ola, aB, ac and ab for the respective pixels A, B, C 
and D in an original image into values corresponding to the 
respective Z-values ZA, ZB, ZC and ZD in the pixels A, B, C and 
D to generate an alpha-plane (in which the alpha-value for each 

25 pixel is set) as shown by E2 . More particularly, the alpha- 
value may increase close to 1.0 as a pixel is farther than the 
focus (or fixation point) of the virtual camera 10 (or as a pixel 
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has its larger difference between the Z-values of th^t pixel 
arid focus). Thus, the synthesis ratio of the defocused image 
increases as the pixel is farther than the focus of the virtual 
camera 10. 

The Z- value may increase as the pixel is farther or closer 
to the virtual camera 10. The synthesis ratio of the defocused 
image may increase as the alpha-value approaches 1.0* (100%) or 
0.0 (0%) . In such a case, the aforementioned formulas (1) , (2) 
and (3) will be used replacing a - 1 with a. 

In this embodiment, as shown by E3 in Fig. 2, the 
alpha- synthesis of the original image with the defocused image 
(e.g., alpha-blending or the like) is carried out based on the 
generated alpha-plane (which is, in a broad sense, alpha-value 
set for each pixel) . Figs . 3A and B show original and defocused 
images obtained through the alpha -synthesis. 

When the alpha - synthesis of the original image with the 
defocused image is carried out based on the alpha-values set 
corresponding to the Z - values (or depth values ) in such a manner , 
a so-called depth of field can be represented such that an image 
located farther than the focus of the virtual camera (which is 
set as a focused point) is generated as a more defocused image. 
Thus, a more real and natural image can be generated to be focused 
depending on the distance from the viewpoint as in the real world, 
unlike the game image of the prior art in which all the objects 
in the screen were focused. As a result, the virtual reality 
for players can highly be improved. 

In addition, this embodiment does not require any 
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calculation based on the positional relationship between each 
of the objects and the viewpoint. Therefore, the depth of field 
can be represented with reduced processing load. It is of course 
that such a calculation may be carried out in the present 
invention. 

2.2 alpha-value setting 

Figs. 4A, 4B and Figs. 5A, 5B illustrate the setting of 
alpha -values depending on the Z- values. _ „ 

As shown in Fig. 4A, for example, areas ARO to AR4 and 
AR1 ' to AR4 ' are sectionalized based on Z - values Zl to Z4 and 
Zl' to Z4 ' (threshold values), alpha-values ceo to a4 and a 
1' to 0L4' are set to these areas ARO to AR4 and AR1 ' to AR4 ' . 

For example, an alpha-value ai may be set for an pixel 
in the area AR1 between Zl and Z2; another alpha- value 0L2 may 
be set for an pixel in the area AR2 between Z2 and Z3 ; still 
another alpha -value #1' may be set for an pixel in the area 
AR1 ' between Zl ' and Z2 ' ; and a further alpha -value a2' may 
be set for an pixel in the area AR2 ' between Z2 ' and Z3 ' . 

The alpha- values set for the respective areas may be set 
based on the following relational expressions: 

ao < a i < <X2 < a.3 < a4 (4) 

ao<ai'<a2'<a3'<a4' (5) 

As will be apparent from these relational expressions (4) 
and (5) , the alpha-value will increase as the pixel thereof is 
farther from the focus (or fixation point) of the virtual camera 
10 . If a is replaced by a - 1 in the relational expressions ( 1 ) , 
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(2) and (3), the alpha-value will reversely decrease as the 
pixel thereof is farther from the focus of the virtual camera 
10. In other words, this embodiment sets the alpha-value such 
that the synthesis ratio of the defocused image increases as 
5 the difference between the focus of the virtual camera 10 and 
the Z-value increases. 

Moreover, this embodiment variably controls thfe range of 
depth of field or def ocusing effect by varying the corresponding 
relationship between- the Z-value and alpha-value ,for each 
10 pixel . 

For example, it is now assumed that a range in which the 
alpha- value becomes smaller than 1.0 (or the synthesis ratio 
of the defocused image becomes smaller than 100%) is defined 
to be a range of depth of field. Thus, the range of depth of 

15 field can be increased when the Z-value is coordinated with the 
alpha- value in such a manner as shown in Fig. 4A. On the other 
hand, the range of depth of field can be decreased when the 
Z-value is coordinated with the alpha -value in such a manner 
as shown in Fig. 4B. 

20 The alpha-value specifying the boundary in the range of 

depth of field is not limited to 1.0, but may be optional. 

If the Z-value is coordinated with the alpha-value as 
shown in Fig. 5A, the def ocusing effect can be decreased while 
if the Z-value is coordinated with the alpha-value as shown in 

25 Fig. 5B, the def ocusing effect can be increased . In other words , 
the alpha-values in the area AR2 (or AR2 ' ) between Z2 and Z3 
(or between Z2 ' and Z3 ' ) are set to 0 . 4 (or decreased def ocusing 
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effect) in Fig. 5A while the alpha-values in the area AR2 (or 
AR2 ' ) are set to 0.7 (or increased defocusing effect) in Fig. 
5B. 

As shown in Fig. 6A, this embodiment performs the 
sectionalization of areas and the setting of alpha- value to the 
respective areas both on the farther and nearer sides to the 
focus. As shown in Fig. 6B, however, the sectionalization of 
areas and the setting of alpha-value to the respective areas 
may be carried out only on the farther side from the focus. 
Alternatively, as shown. in Fig. 6C, the sectionalization of 
areas and the setting of alpha-value to the respective areas 
may be carried out only on the nearer side to the focus. 

The setting of alpha- values as shown in Figs. 6A, B and 
C can be realized through a technique which will be described 
in detail below. 

As shown in Fig. 7A, an object OBI (polygon) having its 
Z-value set to Zl is first drawn in a frame buffer (which is, 
in a broad sense, the drawing area in an original image) . The 
alpha-value for a pixel located on the farther side (which is, 
in a broad sense, the first direction including the nearer side) 
from the object OBI is then updated. In other words, the 
hidden- surf ace removal process (or Z-butter process) based on 
the Z-value is effectively used to update the alpha-value for 
a pixel located farther from the object OBI. 

As shown in Fig. 7B, another object OB2 having its Z- 
value set to Z2 is then drawn in the frame buffer to update the 
alpha-value for a pixel located farther from the object OB2 . 
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Similarly, as shown in Fig. 7C, still another object OB3 having 
its Z -value set to Z3 is then drawn in the frame buffer to update 
the alpha- value for a pixel located farther from the object OB3 . 
By using such a technique, the alpha- values for the pixels 
5 . in the areas AR1, AR2 and AR3 can be set to #1, 0C2 and a3, 
respectively. 

And yet, such a technique provides an advantage that the 
alpha- values can automatically be updated through the 
hidden- surf ace removal function using the Z -buffer only by 
10 drawing the objects on the frame buffer in which the original 
image had been drawn. This enables the desired alpha- value to 
be set to the corresponding pixel in each area with 
substantially reduced processing load. 

Figs. 8A to 9D illustrate the more details of the 
15 alpha-value (or alpha-plane) setting technique . 

As shown in Fig. 8A, the alpha- values in all the areas 
ARO to AR3 and AR1 ' to AR3 ' are first initialized, for example, 
to (111) (where the alpha-values being of 3 bits). 

As shown in Fig. 8B, the Z-value is then set to Zl ' and 
20 an object OBI' (near) having its alpha- value set, for example, 
to (000) is drawn in the frame buffer. Thus, the first bit of 
the alpha-value for a pixel located farther from of or nearer 
to the object OBI' (which is, in a broad sense, the first 
direction) is updated, for example, to 0 (which is, in a broad 
25 sense, first level) . In such a case, however, the bits other 
than the first bit (or bit to be updated) are masked. 

As shown in Fig. 8C, the Z-value is then set to Zl and 



23 



an object OBI having its alpha-value set, for example, to (111) 
is drawn in the frame buffer. Thus, the first bit of the 
alpha- value for a pixel located farther from the object OBI 
(far) is updated, for example, to 1 (which is, in a broad sense, 
5 second level) . In such a case, however, the bits other than the 
first bit are also masked. 

In such a manner, the alpha-value in the area ARO is set 
to (110) while the alpha-values in the other areas are set to 
(111) . Namely, the alpha-value set in the area ARO can be 
10 different from the alpha- values in the other areas. 

As shown in Fig. 9A, an object OB2 ' having its alpha - 
value =(000) is then drawn at Z = Z2 ' to update the second bit 
of the alpha-value for a pixel located farther from the object 
OB2 ' to 0 (with the bits other than the second bit being masked) . 
15 As shown in Fig. 9B, an object 0B2 haying its alpha- value 

= (111) is then drawn at Z = Z2 to update the second bit of the 
alpha-value for a pixel located farther from the object OB2 to 
1 (with the bits other than the second bit being masked) . 

In such a manner, the alpha-value in the area ARO is set 
20 to (100); the alpha-values in the areas AR1 ' and AR1 are set 
to (101); and the alpha-values in the other areas are set to 
(111) . 

As shown in Fig. 9C, an object OB3 ' having its alpha- 
value = (000) is then drawn at Z = Z3' to update the third bit of 
25 the alpha- value for a pixel located farther from the object OB3 ' 
to 0 (with the bits other than the third bit being masked) . 

As shown in Fig. 9D, an object OB3 having its alpha-value 
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= (111) is. then drawn at Z = Z3 to update the third bit of the 
alpha- value for a pixel located farther from the object OB3 to 
1 (with the bits other than the third bit being masked) . 

In such a manner, the alpha -value in the area ARO is set 
5 to (000) ; the alpha-values in the areas AR1 ' and AR1 are set 
to (001); the alpha-values in the areas AR2 ' and AR2 are set 
to (001); and the alpha-values in the areas AR3 ' arid AR3 are 
set to (111) . 

Namely, a pixel having its increased difference of Z- 
10 value relative to the focus of the virtual camera 10 will be 
set to be increased in alpha-value, thus resulting in increase 
of the synthesis ratio of the defocused image. Therefore, the 
alpha-value optimum to the representation of the depth of field 
can be realized through a simple and high- speed process in which 
15 the objects are only drawn masking the particular bits. 

There has described the updating of the alpha-value for 
a pixel located farther from an object merely by drawing that 
object. However, if the image generating system has a function 
of updating the alpha-value for a pixel located nearer to an 
20 object by drawing that object or a function of inverting the 
Z-value, the alpha-value for a pixel located nearer to the 
object may be updated by drawing the object. 

2.3 Generation of defocused image 
25 This embodiment effectively uses the bi-linear filtering 

method (or texel interpolation method) for texture mapping to 
generate the defocused image (Fig. 3B) to be synthesized with 
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the original image (Fig. 3 A) . 

In the texture mapping, the position of a pixel may be 
offset from the position of a texel . 

In such a case, the point sampling method is such that 
5 a color CP (which is, in a broad sense, image information) at 
a pixel (or sampling point) P becomes a color CA at a texel TA 
nearest to the point P, as shown in Fig. 10. 

On the other hand, the bi -linear filtering method is such 
that the color CP at the point P becomes a color obtained by 
10 interpolating colors CA, CB, CC and CD at texels TA, TB, TC and 
TD around the point P. 

More particularly, a coordinate ratio 0:1-0 (0 ^ 0 
^ 1) in the X-axis direction and a coordinate ratio y : 1 - y 
(0 ^ 1) in the Y-axis direction are determined based on 

15 the coordinates of TA to TD and the coordinates of P. 

In this case, the color CP (or output color through the 
bi- linear filtering method) at P can be represented by the 
following formula: 

cp « (i - 0 ) * (i - y ) * CA + 0 * (i - y ) * cb 

20 + (l -/9)x<yxcC + #xyxCD ( 6 ) 

For generating the defocused image, this embodiment aims 
to the fact that the bi- linear filtering method automatically 
interpolates the color in such a manner. 

More particularly, as shown by Fl in Fig. 11, an original 
25 image drawn, for example, in the frame buffer is set as a texture . 
When this texture (or original image) is to be mapped on an object 
through the bi - linear filtering method, the texture coordinates 
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for each vertex in the object are shifted (or slid ox; moved) 
in the rightward and downward direction, for example, by (0.5, 
0.5) . Thus, a defocused image in which the pixel colors in the 
original image spread the surrounding area can be generated 
5 V automatically through the interpolation in the bi-linear 
filtering method. 

Where it is desired to def ocus the entire scene, the shape 
of an object on which the texture (or original image) is mapped 
is set to be identical with that of the scene (defocused area) . 
10 In other words, if the vertex coordinates of the scene are (X, 
Y) = (0, 0), (640, 0), (640, 480) and (0, 480), the vertex 
coordinates in the object are also (X, Y) = (0, 0) , (640, 0) , 
(640, 480) and (0, 480) . 

If in such a case, texture coordinates (U, V) provided 
15 to the vertexes VX1, VX2 , VX3 and VX4 of an object are set to 
be respectively (0, 0), (640, 0), (640, 480) and (0, 480), the 
positions of the scene pixels coincide with the texel positions 
of a texture. Therefore, the image will not be defocused. 

On the contrary, if the texture coordinates (U, V) 
20 provided to the vertexes VX1, VX2 , VX3 and VX4 of the object 
are set to be respectively (0.5, 0.5), (640.5, 0.5), (640.5, 
480.5) and (0.5, 480.5) , the positions of the scene pixels will 
offset from the texel positions of a texture. Therefore, the 
image will be viewed defocused by color interpolation through 
25 the interpolation of the bi-linear filtering method. 

If it is desired to defocus only part of the scene, the 
shape of the object may be identical with that of the defocused 
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area 

As shown by Gl in Fig . 12, this embodiment sets the original 
image as a texture and then shifts it, for example, in the 
rightward and downward direction (or first shift direction) by 
5 0.5 texel. Thus, the texture mapping is performed through the 
bi - linear filtering method to generate a first def ocused image . 
As shown by G2 in Fig. 12, the first def ocused image is then 
set as a texture and shifts it , for example, in the leftward 
~and upward direction (or second shift direction) by 0.5 texel. 
10 Thus, the texture mapping is performed through the bi-linear 
filtering method to generate a second def ocused image. 
Alternatively, a plurality of such processes (shifts in the 
rightward and downward direction and in the leftward and upward 
direction) may be repeated . Thus, a natural def ocused image can 
15 be generated. 

. The principle of generating a def ocused image through the 
interpolation of the bi-linear filtering method will be 
described below. 

It is now assumed that the texture coordinates have been 
20 shifted in the rightward and downward direction by 0.5 texels 
and the texture mapping has been carried out through the 
bi-linear filtering method, as shown in Fig. 13A. In this case, 
0 m y m 1/2 in the above formula (6) . Thus, if it is assumed 
that texels T44, T45, T54 and T55 have their colors C44, C45, 
25 C54 and C55, the color CP44 of a pixel P44 is represented by: 
CP44 = (C44 + C45 + C54 + C55)/4 (7). 
As will be seen from the foregoing, the transformation 



28 



shown in Fig . 13A will cause the color C44 of the texel T44 (which 
corresponds to the original color in the pixel P44 of the 
original not - transformed image) to spread to the surrounding 
pixels P33, P34, P43 and P44 each by 1/4. 
5 . it is thereafter assumed that the image obtained in Fig. 

13A is used as a texture and that the texture mapping is carried 
out through the bi - linear filtering method after the texture 
coordinates thereof have been shifted in the leftward and upward 
direction by 0.5 texels, as shown in Fig. 13B. In this case, 
10 the pixels P33, P34, P43 and P44 of Fig. 13A will correspond 
to the texels T33, T34, T43 and T44 of Fig. 13B, respectively. 
Thus, the color C44, which has spread to the pixels P33, P34, 
P43 and P44 (T33, T34, T43 and T44) each by 1/4 in Fig. 3A, will 
further spread to the four surrounding pixels each by 1/4. As 
15 a result, the color C44 of the original texel T44 will spread 
to the surrounding pixels by 1/16 = 1/4 x 1/4. 

Therefore, the transformations of Figs. 13A and B will 
cause the color C44 (which corresponds to the original color 
of the pixel P44 in the original image drawn in the frame buff er) 
20 to spread to the pixels P33, P34 and P35 by 1/16, 2/16 and 1/16, 
respectively. Moreover, the color C44 will spread to the pixels 
P43, P44 and P45 by 2/16 , 4/16 and 2/16, respectively. The color 
C44 will spread to the pixels P53, P54 and P55 by 1/16, 2/16 
and 1/16, respectively. 
25 As a result, the transformations of Figs. 13A and B will 

apply such a plane filter as shown in Fig. 14A to the original 
image. This plane filter can generate an ideal defocused image 
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for the original image by uniformly spreading the colors of the 
respective pixels in the original image to the surrounding 
pixels 

If the transformations of Fig. 13A and B are twice repeated 
as a set, such a plane filter as shown in Fig . 14 B will be applied 
to the original image. Such a plane filter can generate a more 
ideal defocused image than that of Fig. 14A. 

3. Process of this embodiment 

The detailed process of this embodiment will be described 
in connection with the flowcharts of Figs. 15 and 16. 

The original image (or perspec tively transformed image) 
is first drawn in the frame buffer (step SI) . The technique as 
described in connection with Figs. 8A to 9D is then used to 
prepare an alpha-plane (or alpha - channel ) (step S2) . Then j is 
initialized to 0 (step S3) . 

Subsequently, as shown by Gl in Fig . 12 , the original image 
in the frame buffer is then set as a texture which is in turn 
mapped on a another buff er (step S4) . At this time, the mapping 
is carried out through the bi- linear filtering method after the 
texture coordinates have been shifted by (0.5, 0.5) . 

Subsequently, as shown by G2 in Fig. 12, the image in the 
other buffer is set as a texture which is in turn mapped onto 
the frame buffer (step S5) . At this time, the mapping is carried 
out through the bi- linear filtering method after the texture 
coordinates have been shifted by ( - 0 . 5 , - 0 . 5) . At the same time, 
as shown by E3 in Fig. 2, the alpha-plane generated at the step 
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S2 is used to perform the alpha - blending between the original 
image (Fig. 3A) and the defocused image (Fig.. 3B) . 

Then j is incremented by one (step S6) . It is judged whether 
or not j exceeds M (the number of repeated defocusing 
5 . transformations) (step S7) . If not so, the procedure returns 
to the step S4 . If the j exceeds the M, the procedure is 
terminated. 

Fig. 16 is a detailed flowchart of generating the 
alpha -plane shown in Fig. 15. 

10 As described in connection with Fig. 8A, all the 

alpha- values in the alpha-plane are initialized with 2 m - 2 n_1 
(step S10) . Then i is initialized to n (step Sll) . 

Symbols m and n used herein represent the maximum and 
minimum bits in the prepared alpha-plane, respectively. For 

15 example , if the alpha - value is 8 bits and m and n are respectively 
equal to 6 and 4, the alpha-value of the alpha-plane is 
initialized to (00111000) at the step S10. In other words, the 
initialization at the step S10 will initialize the alpha-value 
so that all the m - n bits are made one (1) with the other bits 

20 being made zero (0) . 

All the R, G, B and alpha -values of the original image 
in the frame buffer are then masked except the i - th bit in the 
alpha-value (step S12) . As described in connection with Fig. 
8B, an object (or polygon) having Z = Zi ' (near) and a = (00000000) 

25 is then drawn in the frame buffer to update the alpha -value of 
a pixel located farther from that object (step S13) . As 
described in connection with Fig . 8C, another object having Z = Zi 
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(far) and a= (11111111) is then drawn in the frame buffer to 
update the alpha -value of a pixel located farther from that 
object (step S14) . 

Then i is incremented by one (step S15) . It is then judged 
5 whether or not i exceeds K (step S16) . If not so, the procedure 
returns to the step S12. If i exceeds K, the process of preparing 
the alpha -plane is terminated. 

4 . Hardware arrangement 

10 A hardware arrangement which can realize this embodiment 

is shown in Fig. 17. 

A main processor 900 operates to execute various 
processings such as game processing, image processing, sound 
processing and other processings according to a program stored 

15 in a CD (information storage medium) 982, a program transferred 
through a communication interface 990 or a program stored in 
a ROM (information storage medium) 950. 

A coprocessor 902 is to assist the processing of the main 
processor 900 and has a product- sum operator and analog divider 

20 which can perform high-speed parallel calculation to execute 
a matrix (or vector) calculation at high speed. If a physical 
simulation for causing an object to move or act (motion) 
requires the matrix calculation or the like, the program running 
on the main processor 900 instructs (or asks) that processing 

25 to the coprocessor 902. 

A geometry processor 904 is to perform a geometry 
processing such as coordinate transformation, perspective 
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transformation, light source calculation, curve formation or 
the like and has a product - sum operator and analog divider which 
can perform high- speed parallel calculation to execute a matrix 
(or vector) calculation at high speed. For example, for the 
5 coordinate transformation, perspective transformation or 
light source calculation, the program running on the main 
processor 900 instructs that processing to the' geometry 
processor 904. 

A data expanding processor 906 is to perform a. decoding 

10 process for expanding image and sound compressed data or a 
process for accelerating the decoding process in the main 
processor 900. In the opening, intermission, ending or game 
scene, thus, an MPEG compressed animation may be displayed. The 
image and sound data to be decoded may be stored in the storage 

15 devices including ROM 950 and CD 982 or. may externally be 
transferred through the communication interface 990. 

A drawing processor 910 is to draw or render an object 
constructed by primitive faces such as polygons or curved faces 
at high speed. On drawing the object, the main processor 900 

20 uses a DMA controller 970 to deliver the object data to the 
drawing processor 910 and also to transfer a texture to a texture 
storage section 924, if necessary. Thus, the drawing processor 
910 draws the object in a frame buffer 922 at high speed while 
performing a hidden- surf ace removal by the use of a Z-buffer 

25 or the like, based on the object data and texture. The drawing 
processor 910 can also perform alpha -blending (or translucency 
processing), depth cueing, mip-mapping, fogging, bi-linear 
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filtering, tri-linear filtering, anti- aliasing, shading and 
so on . As the image for one frame is written into the frame buffer 
922, that image is displayed on a display 912. 

A sound processor 930 includes any multi - channel ADPCM 
5 sound source or the likie to generate high-quality game sounds 
such as BGMs, sound effects and voices. The generated game 
sounds are outputted from a speaker 932. 

The operational data from a game controller 942, saved 
data from a memory card 944 and personal data may externally 
10 be transferred through a serial interface 940. 

ROM 950 has stored a system program and so on . For an arcade 
game system, the ROM 950 functions as an information storage 
medium in which various programs have been stored. The ROM 95 0 
may be replaced by any suitable hard disk. 
15 RAM 960 is used as a working area for various processors . 

The DMA controller 970 controls the DMA transfer among 
processors and memories (such as RAM, VRAM and ROM) . 

The CD drive 980 drives a CD (information storage medium) 
982 in which the programs, image data or sound data have been 
20 stored and enables these programs and data to be accessed. 

The communication interface 990 is to perform data 
transfer between the image generating system and any external 
instrument through a network. In such a case, the network 
connectable with the communication interface 990 may take any 
25 of communication lines (analog phone line or ISDN) or high- 
speed serial interface bus . The use of the communication line 
enables the data transfer to be performed through the INTERNET. 
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If the high-speed serial interface bus is used, the data 
transfer may be carried out between the image generating system 
and any other image generating system or any other game system. 

All the means of the present invention may be executed 
only through hardware or only through a program which has been 
stored in an information storage medium or which is distributed 
through the communication interface. Alternatively/ they may 
be executed both through the hardware and program. 

If all the means of the present invention are .executed 
both through the hardware and program, the information storage 
medium will have stored a program for realizing (or executing) 
the means of the present invention through the hardware. More 
particularly, the aforementioned program instructs the 
respective processors 902, 904, 906, 910 and 930 which are 
hardware and also delivers the data to them, if necessary. Each 
of the processors 902, 904, 906, 910 and 930 will execute the 
corresponding one of the means of the present invention based 
on the instruction and delivered data. 

Fig. 18A shows an arcade game system to which this 
embodiment is applied. Players enjoy a game by controlling 
levers 1102 and buttons 1104 while viewing a game scene 
displayed on a display 1100 . A system board (circuit board) 1106 
included in the game system includes various processor and 
memories which are mounted thereon. Information (program or 
data) for realizing all the means of the present invention has 
been stored in a memory 1108 on the system board 1106, which 
is an information storage medium. Such information will be 
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referred to "stored information" later. 

Fig. 18B shows a home game apparatus to which this 
embodiment is applied. A player enjoys a game by manipulating 
game controllers 1202 and 1204 while viewing a game picture 
displayed on a display 1200. In such a case, the aforementioned 
stored information pieces have been stored in DVD 1206 and 
memory cards 1208, 1209 which are detachable information 
storage media in the game system body. 

"Fig. 18C -shows an example wherein this embodiment is 
applied to a game system which includes a host device 1300 and 
terminals 1304-1 to 1304 -n connected to the host device 1300 
through a network (which is a small-scale network such as LAN 
or a global network such as INTERNET) 13 02. In such a case, the 
above stored information pieces have been stored in an 
information storage medium 1306 such as magnetic disk device, 
magnetic tape device, semiconductor memory or the like which 
can be controlled by the host device 1300, for example. If each 
of the terminals 1304-1 to 1304 -h are designed to generate game 
images and game sounds in a stand-alone manner, the host device 
1300 delivers the game program and other data for generating 
game images and game sounds to the terminals 1304-1 to 1304 -n. 
On the other hand, if the game images and sounds cannot be 
generated by the terminals in the stand-alone manner, the host 
device 1300 will generate the game images and sounds which are 
in turn transmitted to the terminals 1304-1 to 1304 -n. 

In the arrangement of Fig. 18C, the means of the present 
invention may be decentralized into the host device (or server) 
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and terminals. The above information pieces for realising the 
respective means of the present invention may be distributed 
and stored into the inf ormation storage media of the host device 
(or server) and terminals. 

5 Each of the terminals connected to the network may be 

either of home or arcade type. When the arcade game systems are 
connected to the network, it is desirable that each of the arcade 
game systems includes a portable information storage device 
(memory card or portable game machine) which can not only 

10 transmit the information between the arcade game systems but 
also transmit the information between the arcade game systems 
and the home game systems. 

The present invention is not limited to the things 
described in connection with the above forms, but may be carried 

15 out in any of various other forms . 

For example, the invention relating to one of the 
dependent claims may not contain part of the structural 
requirements in any claim to which the one dependent claim 
belongs. The primary part of the invention defined by one of 

20 the independent claim may be belonged to any other independent 
claim. 

It is desirable that the setting of alpha- value is carried 
out through such a technique as described in connection with 
Figs. 6A to 9D. However, the present invention is not limited 
25 to such a technique. For example, the alpha -value may 
continuously be varied depending on the change of the depth 
value (Z-value), rather than the stepwise manner. 
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Although it is desirable that the defocused image to be 
synthesized with the original image is generated through such 
a technique as described in connection with Figs. 11 and 12, 
the present invention is not limited to such a technique. For 
5 example, the defocused image may be generated by synthesizing 
the original image with an image obtained by offsetting the 
original image or by synthesizing an original image in one frame 
with another original image in the previous frame. 

In the invention in which the alpha -value is set as in 
10 Figs. 6A to 9D, the set alpha- value may be utilized to various 
other image drawings, rather than the synthesis between the 
original image and the defocused image. For example, the set 
alpha-value may be used to make an object located farther than 
the viewpoint translucent so that it will merge into the 
15 background. 

The invention in which the defocused image is generated 
through the texel interpolation method is not limited to such 
a technique as described in connection with Figs. 11 to 14B. 
For example, rather than defocusing of the entire scene, a 
20 defocused area smaller than the; scene may be set to defocus the 
original image in that area. 

Although this embodiment has been described as to the 
object located farther than the viewpoint being increased in 
depth value, the present invention is not limited to this. For 
25 example, the present invention may similarly be applied to the 
object located farther than the viewpoint being decreased in 
depth value. 
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The present invention may be applied to any of various 
games such as fighting games, shooting games, robot combat games , 
sports games, competitive games, roll-playing games, music 
playing games, dancing games and so on. 

Furthermore, the present invention can be applied to 
various image generating systems such as arcade game systems, 
home game systems, large- scaled multi-player attraction 
systems, simulators, multimedia terminals, game image 
generating system boards and so on. 
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